1. Field of the Invention
The invention described herein comprises a system, method, and computer program product for single-handed input of alphanumeric characters and associated commands on a variety of communication devices, including mobile phones, PDA's and computers, and the like.
2. Background Art
The advent of worldwide communication networks and mobile computing devices, such as phones, PDA's, wireless pagers, etc., has stimulated immense research into and development of new and more efficient means for fast, reliable text input. The continuing development of Short Message Services (SMS) and of even more complex MultiMedia Services (MMS) has made the issue of text input increacingly important. A successful, universally usable-method of text input is an essential requirement for the success of emerging technological advances in mobile computing.
To ensure the efficient operation of any new method of input, a number of important problems should be addressed, that are simultaneously required for its efficient operation. Morse code (single and double keyer methods) and touch-typing on QWERTY keyboards have been universally adopted and used successfully for generations. By examining some important parameters of text input shared by these methods, we can establish a conceptual framework within which to discuss the existing techniques.
It appears that they hold in common the following features:
Universal: Both allow multiple languages to be accessed. Morse code quickly became a worldwide method of communication. The QWERTY layout and mechanism have been adopted throughout the world with minor differences dependent on the language used.
Ergonomic: Their use is related to normal coordinated movement involving simultaneously overlapping actions, which are both timed and sequential.
Automatic: They allow for automatic skill to develop that does not require conscious monitoring, and therefore places few cognitive demands on the user after the initial learning period.
Independent of Vision: For both systems, after skilled targeting or timing have developed, visual monitoring of the input and confirmation of output is not constantly required.
Efficient and Accurate: High speed of input with a low error rate is achieved by both methods.
Economical: The technology involved is inexpensive to produce and is available to a wide range of consumers.
Multi-functional: These systems allow the user to focus on parallel activities as attentional requirements are not overly demanding. Morse code is a particular case in point, since it requires only one hand and can be used in a variety of situations.
The features listed above provide the context in which to discuss text entry techniques. A number of alternative approaches have been developed in the search for successful methods of text input in mobile environments. Text entry using soft keyboards and reduced keyboards with fewer keys have been developed by a number of commercial enterprises. Input methods include sequential, coded entry as well as numerous variations of chord keyboards, where entry is by combinations of simultaneously activated keys. A close examination of the movement patterns required by these methods reveals an important ergonomic design flaw common to them all that underlies the difficulty their users experience
Soft keyboard input uses a single activator (stylus or finger) to select directly from a full-alphabet display. This is similar to typing with a single finger. Although the target for each character remains constant, the actual trajectory of the stylus moving to any one key will depend on the location of the previously activated key. Each key therefore has multiple trajectories associated with its activation. The need to visually guide the stylus visually results from this one-to-many relationship. It is therefore practically impossible to learn the multiple pathways to a key and reproduce them by touch alone.
Much research has occurred aimed at facilitating this jumping requirement by designing the most efficient and ergonomic layout to minimize the degree of movement needed. This research uses Fitt's equation relating distance traveled and time taken for moving from target to target. (See D. Hughes et al., “Empirical bi-action tables: A tool for the evaluation and optimization of text input systems. Application I: Stylus keyboards” Human-Computer Interaction, 17, (2002), Lawrence Erlbaum Associates, Inc. Hughes et al. relate the inter-key time parameters of a defined keyboard matrix to various alphabetic layouts. They optimize the layout using standard heuristic algorithms (e.g., the ant algorithm) to define the most efficient layout. Using a digraph table of 26 letters plus space, Hughes et al. compute the most effective layout. Because their concern is the physical tapping time required for accessing sequential characters, their focus is on arranging the letters to maximize typing speed.
However, increased speed when tapping with one finger or a stylus can also result in ever increasing tension and rigidity of muscles and movement. This method does not take advantage of the capacity for coordinated overlapping movements naturally available to a five fingered hand. It also requires constant visual monitoring and therefore precludes touch-typing. Because of its attentional demands, using this method makes it difficult to engage simultaneously in other parallel activities, such as walking. It is inherently dependent on sophisticated technologies like touch screens.
The number of keys on a keyboard can be reduced, and sequences of keystrokes programmed to represent different outputs. These methods are described as two-stroke or sequential input. The reduction can be applied to both physical as well as soft keyboards (e.g., onscreen). An example of the former is U.S. Pat. No. 6,348,878 (the '878 patent) to Tsubai, which states that: “A keyboard layout for a one-handed keypad having fifteen alphabetic keys. Each key has a primary letter and a secondary letter. The primary letter is keyed by solely striking the key, while the secondary letter requires striking a secondary key first or simultaneously with the primary alphanumeric key. The layout placement minimizes finger travel and keystrokes to generate the most common letters and digraphs in the English language.”
According to the '878 patent, each letter of the alphabet requires activation of a particular sequence of fingers. Individual fingers move in two directions to press a key, both up and down and across the three levels of keys from upper, to home, to lower. It is apparent that in repeated typing of a particular letter in different word contexts, the position of each finger will differ depending on its involvement in typing a prior letter. Multiple trajectories and combinations of trajectories will be used to type a particular letter. Thus, although each letter is coded by a single key sequence, the movements used in activating the sequence do not have a one-to-one relationship to the codes.
This inherent complexity goes unnoticed in the usual description of this and numerous other methods involving reduced keyboards. It may be noted here that the standard method of touch-typing on a QWERTY keyboard is designed precisely to reduce the number of alternate movements required by each finger. Maintaining a strict obedience to the rules of touch-typing is a means of reducing the complexity of movement required. Some fingers (the index fingers) still need to access at least six keys each. It is this remaining complexity that ensures that most individuals need a period of learning and practice to develop skilled “eyes-free” typing.
MessagEase, from Exideas Inc., is an example of a reduced soft keyboard. It is described as a 12 key two-keystroke method of text entry using a telephone keypad or Palm Pilot PDA, where a virtual onscreen keyboard version is accessed with a stylus. The 9 most common letters are entered by double pushes of one of the 9 number keys. A further 8 letters are entered by sequential two-key pushes from the periphery to the center key (the number 5) and another 8 emanate from the center key outwards to the peripheral keys. Extra characters are coded on the remaining 3 keys. There is no timeout provided. As a result, if a key is pushed in error it is registered as part of the sequence. Should a key be missed in error, the subsequent code will also be incorrect. To avoid this, continual visual monitoring is required for avoidance and correction of errors.
Two sequential activations are taken to represent a single code (e.g., alphanumeric output). Again, the distinction between coding and the actual finger movements needed to create the codes is important. One finger is typically used to input the code. The resultant movement is therefore a single actuator jumping from one location to another. The coding requires sequential activation but the movement used is similar to the repetitive activation of a single key.
This issue is not eliminated even if all fingers are used, as in U.S. Pat. No. 4,333,097 (the '097 patent) to Buric et. al., titled “Visual Display Terminal Without Finger Repositioning.” The '097 patent discloses a keyboard of 10 keys, one per finger, to eliminate the need to reposition the fingers and also reduce the targeting requirements. Selected characters are displayed on a screen, horizontally above and below a default list of ten characters, and the thumb keys move the display up or down to the next layout. Frequency of occurrence of characters in English, as well as relative strength of fingers, are factored into the selection of characters. More frequently used characters require fewer key inputs. Note that the exact sequence of key pushes to get to a character will depend on the position of the display as determined by the previous letter. Since a multiple movement sequence is possible for each character, one must determine by visual inspection where the display was positioned in order to work out the appropriate sequence of key pushes. Thus, although all ten fingers are involved, it is still not possible to touch type using this method.
Chord keyboards present a similar problem. They have a reduced number of keys and use combinations of keys to code the output characters. Tables of finger combinations are provided that appear to represent a single combination for each letter. In chord keyboards the moment of activation of the resultant character output in chord keyboards must be coincident with the release of the keys, as only then is it possible to determine which accumulated combination of keys was selected. The consecutive combinations of chords can only be input at the speed attainable by the repeated finger in common to a number of chords. For example, if a series of chords consist of 123-245-23 then the 2 is repeating.
Some chord methods also include “latching”, where a finger common to two consecutive chords is held while changing from one to the next. Presented as a tip to increase speed by reducing the number of key presses, this technique has inadvertent consequences, which adversely affect the relationship of movements to codes. Holding down a common finger creates many alternative fingerings for the same letter, which again makes the fingering of one letter dependent on the preceding letter. The unnoticed complexity of the Twiddler approach, for example, (Twiddler User Manual Rev 1.6, Handykey Corporation, Page 14–15 (2000)) lies in the problem of selecting which finger to release while simultaneously pressing the required finger. The Twiddler “Speed Tip” approach is reproduced in FIG. 20. Based on this Twiddler approach:
a) p=24 i=23 n=25, with a release order of 4-3-5, therefore i=hold 2, exchange 4 and 3 in that order, and release 3.
b) n=25 i=23 p=24, with a release order of 5-3-4, therefore i=hold 2, exchange 5 and 3 in that order and release 3.
Care must be taken to control the order of movements. For example, if the order is reversed in (a), and the 3 is placed before the 4 is released, a new, different code representing a combination of all three fingers will be created. A single “i” is also produced by placing 2 and 3, and releasing both in any order.
The Twiddler approach uses 12 keys accessed by four fingers, so additional complexity comes from the necessity to select which of three rows of keys ((L)eft,(M)iddle or (R)ight) to use. Considering the permutations of release sequences created by the possible preceding 26 letters, and further compounding the release sequences with three targets for each finger, it is apparent how extremely demanding this method really is. A list of finger codes that usually accompanies chord keyboards, is therefore unintentionally, but inherently an over-simplification. No quick method of referring to unknown codes or forgotten key sequences appears available for real time access. The attraction of this method lies in the reduced number of keys required, but it is the hidden complexity of movement it demands (rather than the coding) that explains why, over a 150 year period, this method has never been widely used.
The recent widespread development and use of miniaturized QWERTY keyboards, as in the Blackberry PDA (Research In Motion Limited, Waterloo, Ontario, Canada), raises similar ergonomic issues. The primary motivation for this method is familiarity with the QWERTY layout. However, the original ergonomics of the QWERTY keyboard are disrupted owing to the small size of the miniaturized versions' keys, making the development of automaticity—the most significant feature of the full-size keyboard—practically impossible. Under constant visual guidance, and while holding the device with both hands at a distance sufficient to read the tiny letters on the keys and the small text output, two thumbs (the largest fingers) are required to push these very small keys Touch typing was never envisaged for the Blackberry PDA, nor was extensive text input, owing purely to the physical demands made on single fingers.
It is of interest to note that the only research available to date on the use of thumb keyboards begins to show some awareness of the significance of true sequential input. “The time between keystrokes when using one thumb to repeatedly type the same key is tREPEAT. When using two thumbs to repeatedly alternate between two keys, the keystroke rate almost doubles because the movement of the two thumbs overlaps.” (I. S. MacKenzie et al., “A model of two-thumb text entry,” Proceedings of Graphics Interface 2002, (Toronto: Canadian Information Processing Society, pp. 117–124 (2002).)
In summary, the methods described above lack some or all of the essential characteristics for efficient text input outlined in the opening statement. Any method requiring constant visual attention or complex variable sequences of movements for each character, using only one or two fingers of the hand, precludes the development of an automatic, cognitively undemanding method of text input in a mobile, multi-functional environment. Hence, a need exists for an economical system and method for character entry in multiple languages, which can be performed automatically, with low cognitive demands and in a multiplicity of environments.